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ABSTRACT 

This report presents the results of the 2002 assessment in 
writing from the National Assessment of Educational Progress for the nation 
and for the participating states and jurisdictions. Comparisons are made to 
performance in the national assessment of 1998 at grades 4, 8, and 12. 
Comparison data are given within and across participating states and 
jurisdictions for 2002 at grade 4 and for both 1998 and 2002 at grade 8. 
Student performance is reported in terms of average scale scores on the NAEP 
writing scale and by the percentages of students who attained the achievement 
levels set by the National Assessment Governing Board (NAGB) . In addition, 
the report presents percentile distributions and demographic subgroup results 
for the nation, including results by gender, race/ethnicity, student 
eligibility for free/reduced-price school lunch. Title I, parents* highest 
level of education, type of school, and type of school location. For 
participating states and jurisdictions, performance results for subgroups 
defined by gender, race/ethnicity, and student eligibility for free/reduced- 
price school lunch are presented. The report also includes sample assessment 
writing tasks and examples of student responses in narrative, informative, 
and persuasive writing modes. Scoring guides for each of the writing modes at 
grades 4, 8, and 12 as well as rationales for scores of student responses are 

provided. Appendices include information on national and state samples, 
school and student participation rates, participation and accommodation of 
students with disabilities and/or limited-English-prof icient students, 
subgroup percentages, and state-level contextual variables. (RS) 
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What is The Nation^ Report Cord? 

THE NATION’S REPORT CARD, the National Assessment of Educational Progress (NAEP), is a nationally 
representative and continuing assessment of what America’s students know and can do in various subject areas. 

Since 1969, assessments have been conducted periodically in reading, mathematics, science, writing, history, 
geography, and other fields. By making objective information on student performance available to policymakers at 
the national, state, and local levels, NAEP is an integral part of our nation’s evaluation of the condition and progress 
of education. Only information related to academic achievement is collected under this program. NAEP guarantees 
the privacy of individual students and their families. 

NAEP is a congressionally mandated project of the National Center for Education Statistics, within the Institute 
of Education Sciences of the U.S. Department of Education, The Commissioner of Education Statistics is 
responsible, by law, for canying out the NAEP project through competitive awards to qualified organizations. 

In 1988, Congress established the National Assessment Goveming Board (NAGB) to oversee and set policy for 
NAEP. The Board is responsible for: selecting the subject areas to be assessed; setting appropriate student 
achievement levels; developing assessment objectives and test specifications; developing a process for the review of 
the assessment; designing the assessment methodology; developing guidelines for reporting and disseminating 
NAEP results; developing standards and procedures for interstate, regional, and national comparisons; determining 
the appropriateness of all assessment items and ensuring the assessment items are free from bias and are secular, 
neutral, and non-ideological; taking actions to improve the form, content, use, and reporting of results of the National 
Assessment; and planning and executing the initial public release of National Assessment of Educational Progress 
reports. 
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xecutive Summary 




The National Assessment of Educational Progress (NAEP) 
is an ongoing nationally representative sample survey of 
student achievement in core subject areas. Authorized by 
Congress and administered by the National Center for 
Education Statistics (NCES) within the Institute of 
Education Sciences of the U.S. Department of Education, 

NAEP regularly reports to the public on the educational 
progress of fourth-, eighth-, and twelfth-grade students. 

This report presents the results of the NAEP 2002 writing 
assessment for the nation at grades 4, 8, and 12 and for 
participating states and other jurisdictions at grades 4 and 8. 

Assessment results are described in terms of their average 
writing score on a 0—300 scale and in terms of the 
percentage of students attaining each of three achievement 
levels: Bask, Proficient, ■and Advanced. 

The achievement levels are performance standards adopted 
by tlie National Assessment Governing Board (NAGB) as 
part of its statutory responsibilities. The achievement levels 
are a collective judgment of what students should know and 
be able to do for each grade tested. As provided by law, 

NCES, upon review of a congressionaUy mandated 
evaluation of NAEP, determined that the achievement levels 
are to be considered developmental and should be 
interpreted with caution. Plowever, both the Commissioner 
and the Board believe that these performance standards are 
useful for understanding trends in student achievement. They 
have been widely used by national and state officials, as a 
common yardstick of academic performance. 
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The results presented in this report are 
based on representative samples of students 
for the nation and for participating states 
and other jurisdictions. Approximately 
276,000 students from 11,000 schools were 
assessed. The national results reflect the 
performance of students attending bodi 
public and nonpublic schools, while the 
state and jurisdiction results reflect only the 
performance of students attending public 
schools. Information about writing achieve- 
ment for students in selected urban school 
districts is presented in the NAEP 2002 
Trial Urban District Assessment of writing.^ 

In addition to providing average scores 
and achievement level performance in 
writing for the nation and states and odier 
jurisdictions, this report provides results for 
subgroups of students defined by various 
background characteristics. A summary of 
major findings from the NAEP 2002 
assessment is presented on die following 
pages. Comparisons are made to national 
results from the 1998 assessment. The 
NAEP 1998 writing assessment was not 
administered at the state/jurisdiction level 
at grade 4; therefore, state-level compari- 
sons are presented only for grade 8. 

Changes in student performance across 
years or differences between groups of 
students in 2002 are discussed only if they 
have been determined to be statistically 
significant at the 0.05 level. 



Overall Writing Results for 
the Nation and the States 

Writing Results for the Nation 

■ Students’ average scores on the NAEP 
writing assessment increased between 
1998 and 2002 at grades 4 and 8. How- 
ever, no significant change was detected 
in die performance of twelfth -graders 
between the two assessment years. 

■ Fourth-grade writing scores at the 10th 
to the 90th percentiles increased between 
1998 and 2002. This means that the 
performance of high, middle, and low 
performing students improved between 
the two years. Gains were observed 
among the middle- and higher-perform- 
ing students at grade 8. At grade 12, only 
the score at the 90th percentile increased 
since 1998, while scores at die 10th and 
25th percentiles were lower in 2002. 

■ In 2002, between 24 and 31 percent of 
die students in each of the three grades 
performed at or above the Proficient level. 
Fourth- and eighth-graders made overall 
gains since 1998 in reaching the Proficient 
level. There was no significant change 
detected in the percentage of twelfth- 
graders at or above Proficient; however the 
percentage of twelfth-graders at or above 

decreased since 1998. 



^ Lutkus, A. D, Daaiie, M. C, Weiiier, A. W, aud Jiii, Y. (fortlicoiniiig). The Nation's Re/)ort Card: Trial Urban District 
Assessntenty Writing 2002. Wasliiiigtoii, DC: US. Department of Education, Institute of Education Sciences, 
National Center for Education Statistics. 
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Writing Results for the States 
and Other Jurisdictions 

Results from the 2002 assessment are 
reported for 48 states and other jurisdictions 
at grades 4, and 47 states and other jurisdic- 
tions at grade 8. Results are reported only 
for public-school students at the state or 
jurisdiction level. 

At grade 4 

■ In 2002, fourth-grade average scores were 
higher than the national average score in 
17 jurisdictions, and lower than the 
national average in 22 jurisdictions. 

■ Connecticut, Massachusetts, and Dela- 
ware were among the highest performing 
jurisdictions at grade 4. The average 
writing scores in Connecticut and Massa- 
chusetts were higher than in any of the 
other participating jurisdictions. Massa- 
chusetts was only outperformed by 
Connecticut. Students in Delaware were 
only outperformed by students in Con- 
necticut and Massachusetts and had 
higher scores than the other participating 
jurisdictions except New York. 

At grade 8 

■ Of the 36 jurisdictions that participated 
in both the 1998 and 2002 eighth-grade 
writing assessment, 16 showed score 
increases in 2002 and none showed a 
significant decrease. 

■ The percentage of eighth-graders at or 

above increased in 17 jurisdic- 

tions and decreased in 1 jurisdiction since 
1998. 

■ Connecticut, Department of Defense 
domestic and overseas schools, Massa- 
chusetts, and Vermont were among the 
highest performing jurisdictions at grade 8. 



National and State 
Writing Results for 
Student Subgroups 

In addition to overall results for the nation 
and for the states and jurisdictions, NAEP 
reports on the performance of various 
subgroups of students. Observed differ- 
ences between student subgroups in N AEP 
writing performance most likely reflect the 
interaction of a range of socioeconomic 
and educational factors not addressed in this 
report or by NAEP. 

National Results 
Gender 

■ The average scores of male and female 
fourtli- and eighth-graders were higher in 
2002 than in 1998; however, at grade 12, 
the average scores for male students 
declined. 

■ The percentages of female students 
performing at or above Proficient increased 
since 1998 at all three grades, and the 
percentage of male students performing 
at or above Proficient increased at grades 4 
and 8. 

■ In 2002, female students had higher 
average scores than male students at all 
three grades. 

■ In 2002, females outperformed males 
on average by 17 points at grade 4, 21 
points at grade 8, and 25 points at grade 
12. The decline in the average score for 
male twelftli -graders between 1998 and 
2002 resulted in an increase in tlie gap 
between male and female students. 
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Race/Ethnicity 

■ At grades 4 and 8, White, Black, and 
Hispanic students had higher average 
writing scores in 2002 than in 1998. 

■ The percentages of students performing 
at or above Proficient increased since 1998 
among White, Black, Hispanic, and 
Asian/Pacific Islander students at grade 4 
and among White, Black, and Hispanic 
students at grade 8. 

■ At grade 4, Asian/Pacific Islander stu- 
dents outperformed all other groups in 
2002, and White students outperformed 
Black, Hispanic, and American Indian/ 
Alaska Native students. At grade 8, 

White and Asian /Pacific Islander stu- 
dents scored higher on average than 
Black, Hispanic, and American Indian/ 
Alaska Native students. At grade 12, 
White and Asian/Pacific Islander stu- 
dents scored higher on average than 
Black and Hispanic students, and 
Hispanic students had higher scores 
than Black students. 

■ In 2002, the score gap between White 
and Black fourth-graders was smaller 
than in 1998. 



Eligibility for 

free/ Reduced-Price Lunch 

The program providing free/ reduced-price 
lunch is administered by the U.S. Depart- 
ment of Agriculture (USDA) for children 
near or below the poverty line. Eligibility is 
determined by die USDAs Income Eligibil- 
ity Guidelines (http://www.fns.usda.gov/ 
cnd/IEGs&NAPs/IEGs.htm). 

■ Average fourth- and eighth-grade writing 
scores in 2002 were higher than in 1998 
for students who were eligible for free/ 
reduced-price lunch, as well as for those 
who were not eligible. 

■ The percentages of fourth- and eighth- 
graders at or above Proficient v/ere higher 
in 2002 than in 1998 for students who 
were eligible and those who were not 
eligible for free/reduced-price lunch. 

■ In 2002, die average writing score for 
students who were eligible for free/ 
reduced-price lunch was lower than that 
of students who were not eligible at all 
three grades. 

Title I Participation 

Tide I is a federally funded program that 
provides educational services to children 
who live in areas with high concentrations 
of low- in CO me families. Due to recent 
changes in how the program is adminis- 
tered, comparisons to previous assessment 
year results are not available. 

■ In 2002, students at all three grades who 
attended schools that participated in Title 
I, had lower average writing scores than 
students who attended schools drat did 
not participate in Title I. 
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Parents^ Level of Education 

(reported by students) 

■ There was a positive relationship 
between higher levels of parental 
education as reported by students arid 
student achievement: for both eighth- 
and twelfth-graders, the higher the 
parental education level, the higher the 
average writing score. (Information 
about parental education was not 
collected at grade 4.) 

Type of School 

■ The average writing scores for fourth- 
and eighth-grade public-school students 
were higher in 2002 than in 1998. 

■ In 2002, at all three grades, students who 
attended nonpublic schools had higher 
average writing scores than students who 
attended public schools. At grade 8, 
students who attended Catholic schools 
had higher scores than those attending 
other nonpublic schools. 

Type of School Location 

■ Students in urban fringe schools had 
higher average writing scores than their 
peers in central city schools and rural 
schools at all three grades. Fourth- and 
eighth-grade students in rural schools had 
higher scores than their peers in central 
city schools, while the reverse was true at 
grade 12. 



State and Jurisdiction Results 
Gender 

■ At grade 8, average scores were higher in 
2002 than in 1998 for both male and 
female students in 12 jurisdictions, for 
female students only in 1 jurisdiction, and 
for male students only in 2 jurisdictions. 

■ In 2002, females had higher average 
scores than males in all the participating 
jurisdictions at both grades 4 and 8. 

Race/Ethnicity 

■ At grade 8, average scores increased since 
1998 for White students in 15 jurisdic- 
tions, for Black students in 9 jurisdictions, 
for Hispanic students in 4 jurisdictions, 
and for students classified as Other in 1 
jurisdiction. 

■ Score increases were observed for two or 
more racial/ethnic subgroups of eighth- 
graders in the following jurisdictions: 
Arkansas, Delaware, Florida, Louisiana, 
Maryland, Missouri, North Carolina, 
South Carolina, and Washington. 

Free/Reduced’Price Lunch 

® At grade 8, average scores increased since 
1998 for both those students who were 
eligible for free/reduced-pricc lunch and 
those who were not eligible in 11 jurisdic- 
tions, for eligible students in 1 jurisdic- 
tion, and for students who were not 
eligible in 4 jurisdictions. 
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Introduction 



Writing is a fundamental skill for individuals and for 
civilizations. Writing enables us to record and reflect on our 
experiences, to communicate with others, and to preserve a 
common culture. In our democratic society, writing is a 
central form of discourse. A healthy and civil society requires 
citizens who are able to state a case carefully and to reason 
with others persuasively. Thus, writing has always been an 
important feature of school curricula from the early 
elementary grades through high school and post-secondary 
education. In a technology-based, electronically dependent 
economy, the ability to write clearly is a critical skill for 
advancing knowledge, enhancing competence, posing new 
ideas, and making those ideas comprehensible to an 
information-dependent citizenry.* 

The National Assessment of Educational Progress 
(NAEP) reports on writing assessments are an important 
source of information on students’ writing achievement. 

This report presents major results from the NAEP 2002 
writing assessment of the nation’s fourth-, eighth-, and 
twelfth-grade students. In addition, it provides results for 
fourth- and eighth-grade students in states and other 
jurisdictions that participated in the 2002 assessment. 

Finally, the report compares students’ 2002 performance to 
their performance in 1998. The report is intended to inform 
educators, policymakers, parents, and the general public 
about students’ achievement in writing. 

^ Scardamalia, M. and Bereiter, C. (1991). Literate Expertise. In K. A. Ericsson and J. 

Smitli (Eds.), Toumd a General Theory of Expertise: Prospects and Limits, pp. 172—19. 

New York: Cambridge University Press. 
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Overview of the 2002 
National Assessment of 
Educational Progress 
in Writing 

For more than 30 years, NAEP has regularly 
collected, analyzed, and reported valid and 
reliable information about what American 
students know and can do in a variety of 
subject areas. As authorized by the U.S. 
Congress, NAEP assesses representative 
national samples of fourth-, eighth-, and 
twelfth-grade students. 

Since 1990, NAEP has also assessed 
representative samples of fourth- and 
eighth-grade students in states and other 
jurisdictions that participate in the NAEP 
state-by-state assessments in several sub- 
jects. NAEP is administered and overseen 
by the National Center for Education 
Statistics (NCES), in the U.S. Department 
of Education’s Institute of Education 
Sciences (lES). 

The content of all NAEP assessments is 
determined by subject-area frameworks that 
are developed by the National Assessment 
Governing Board (NAGB). The framework 
for the NAEP writing assessment guided 
development of the assessment that was 
first administered in 1998 and most recently 
in 2002. 

The assessment was first given nationally 
to fourth-, eighth-, and twelfth-graders in 
1998. State-level assessments using the same 
instrument as that used nationally were 
conducted only at grade 8 in 1998. 



The 2002 assessment was conducted at 
grades 4, 8, and 12 nationally, and at grades 
4 and 8 within the states and other jurisdic- 
tions that participated in the state-level 
assessment. Throughout this report, na- 
tional results from die 2002 assessment are 
compared to those from 1998 at all three 
grades. Comparisons of results for states 
and other jurisdictions that participated in 
both the 1998 and 2002 assessments at 
grade 8 are also presented. 

Framework for the 1 998 and 
2002 Writing Assessments 

The NAEP 1998 writing framework is the 
blueprint that has specified the content and 
guided the development of the 1998 and 
2002 writing assessments.” The framework 
establishes the assessment objectives and 
provides direction for the kinds of writing 
tasks to be included in the instrument. The 
framework is a product of a nationwide 
process involving many parties concerned 
about writing education, including teachers, 
state education officials, subject-area special- 
ists, researchers, and representatives of the 
general public. This effort was managed by 
the Center for Research on Evaluation, 
Standards, and Student Testing (CRESST), 
under the direction of NAGB. NAGB also 
contracted with ACT to provide detailed 
guidelines for tiie kinds of writing tasks to 
include in the assessment. 



“ National Assessment Governing Board. Writing Framework and Speciftcations for the 1998 National Assessment of 
Educational Progress. Wasliiiigton, DC: Audior. 
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The NAEP writing framework, informed 
by current research and theory, emphasizes 
that writing addresses a variety of purposes 
and audiences. The framework discusses 
three purposes for writing: narrative 
(telling a story), informative (informing 
tlie reader), and persuasive (persuading the 
reader). To ensure that NAEP writing 



assessments reflect the genres receiving the 
most instructional emphasis, the framework 
prescribes that NAEP writing tasks focus 
on these three purposes at all three grade 
levels (4, 8, and 12).^ Descriptions 
of narrative, informative, and persuasive 
writing appear in figure 1.1. 



Figure 1 .1 Descriptions of the three purposes for writing in the NAEP writing assessment 



Purposes for Writing 



Norrotive writing Narrative writing encourages writers to incorporate their imagination and creativity in the production of 
stories or personal essays. At its best, narrative writing fosters imagination, creativity, and speculation 
by allowing writers to express their thoughts and emotions, and offers an opportunity for writers to 
analyze and understond their actions and those of others. 

Hie narrative tasks included in the NAEP 2002 writing assessment osked students to write many kinds of 
stories (most fiction, some nonfiction). Some of the tasks asked students to write in response to 
photographs, drawings, cartoons, poems, or stories (provided with the assessment). 

Informative writing In informative writing, the writer provides the reader with information. Informative writing may involve 
reporting on events or experiences or analyzing concepts and relationships. When used os a means of 
exploration, informative writing helps both the writer and the reader to learn new ideas and to reexamine 
old conclusions. 

Informative tosks in the NAEP 2002 writing assessment asked students to write on specified subjects 
using many kinds of information, such as newspaper articles, chorts, photographs, or reported diologues 
(provided with the assessment), os well os their own knowledge. Students could write in a voriety of 
formots, such os reports, newpaper articles, and letters. 



Persucsive writing Persuosive writing seeks to persuade the reoder to take action or to bring about change. This type of 
writing involves a clear awareness of whot orguments might most affect the audience being oddressed. 
Writing persuasively also requires the use of such skills as analysis, inference, synthesis, and evaluation. 

Persuasive tosks in the NAEP 2002 writing assessment osked students to write letters to the editor or to 
friends, to refute arguments, or to take sides in o debate. 



SOURCE Notional Assessment Governing Soori lil/h'f/ng framework am/5per/rrro//om hr the 1998 Nalional Jiss^smenl oUdmtional Progrest Wosfungton, DC: Author. 



^ Oil die importance of specifying purpose iu Writing instruction, see Oliver, E. (1989). Effects of Assignment on 
Writing Quality at Four Grade Levels. English Quarterly 224—32. 

Gentile, C. A., Martin- Relirmami,J., and Kemiedy, J. H. (1995). Windows into the Classroom: NAEP’s 1992 Por^olto 
Stud^ (NCES 95-035). Wasliington, DC: US. Dept, of Education, Office of Educational Research and Develop- 
ment, National Center for Education Statistics. 

Applebee, A. N., Langet, J. A., Jenkins, L., Mullis, I. V S., and Foertsch, M. (1990). Learning to Write in Our Nation's 
Schools: Instruction and Achievement in 1988 at Grades 4, 8, and 12, Princeton, NJ: Educational Testing Service. 
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As the framework notes, the purposes 
for writing are not always completely 
discrete. For example, a narrative essay may 
make a persuasive moral or ethical point, 
and a letter to an editor or congressional 
representative may include pertinent facts 
and information. In fact, many of the 
students whose writing received high ratings 
used integrated forms of presentation. 

The professional raters who evaluated the 
student responses were instructed not to 
penalize such blended presentations. 



The emphasis on each purpose for 
writing varies from grade to grade to match 
the differing levels of student development 
and instructional focus. The assessment 
emphasized narrative writing for fourth- 
graders, gave comparable weight to all three 
purposes for eighth-graders, and stressed 
persuasive writing for twelfth-graders. Table 
1.1 shows both the percentage and actual 
number of tasks for each writing purpose at 
each grade level in the 2002 assessment. 
These distributions match the target per- 
centages established by the framework. 



Toble U Distribution of writing tasks, by purpose for writing, in the NAEP 2002 writing assessment, 
grades 4, 8, ond 1 2 



Purpose for 
writing 



Grade 4 



Narrotive 

Informotive 

Persuosive 



Grade 8 



Narrative 

Infarmative 

Persuasive 



Percentage 
af tasks 



40 

35 

25 



35 

35 

30 



Number 
af tasks 



7 

5 



7 

7 

6 



Grade 1 2 



Narrative 25 5 

Informative 35 7 

Persuasive 40 8 



SOUfiCE: Ui. Department of Education, Institute of Education Sconces, Notional Cutter br Education Statistics, Notional Aaessment of Educational Progress (NMP), 2002 Writing Assessment. 
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In addition to specifying the percentage 
of tlie assessment diat should be devoted to 
each writing purpose, the framework speci- 
fies several elements of writing that should 
pertain to writing tasks across the assess- 
ment. When constructing writing assess- 
ment tasks, test developers consider various 
aspects of writing that are important for 
motivating student engagement in die 
assessment tasks. 

A writing task is usually a short text or 
visual stimulus, posing a situation, concern, 
or topic about which students are asked to 
write under a stated time constraint. The 
2002 assessment used many tasks that 
specified the writer's audience. Some stu- 
dents were asked to write, for example, a 
letter to a friend or to a school board. 
Students also had opportunities to write in a 
variety of forms, such as essays, letters, 
reports, and stories. Writing tasks may have 
used any of a variety of stimuli to evoke 
written responses, including photographs, 
cartoons, drawings, newspaper articles, 
letters, or literary works, such as poems or 
stories. In addition, students received a brief 
brochure with suggestions for planning and 
revising their writing. 

To meet die framework's objective that 
students value writing as a communicative 
activity, background questions on the 
assessment asked students about their view 
of themselves as writers and their writing 
practices at home and at school. Data for 
these background questions are available on 
the NAEP web site (http://nceS;ed.gov/ 
nationsrepor tear d/naep data). 



The 2002 NAEP Writing 
Assessment Instrument 

So that the assessment reflects the NAEP 
writing framework and expert perspectives 
on the measurement of writing, it under- 
goes stringent review by teachers, teacher 
educators, state officials, and measurement 
specialists during the development process. 
All components of the assessment are 
evaluated for curricular relevance, develop- 
mental appropriateness, fairness, and adher- 
ence to the framework and test specifica- 
tions. The 2002 writing assessment included 
twenty 25-minute tasks each at grades 4, 8, 
and 12. 

To minimize the burden on any one 
student, NAEP uses a procedure referred to 
as matrix sampling, in which different 
students at any given grade are administered 
only a small portion of the entire assess- 
ment. At each grade, students received test 
booklets with two 25-minute tasks. A 
representative sample of students at each 
grade received each task, and the results 
were combined to produce average group 
and subgroup results based on the entire 
assessment. In addition to the writing tasks 
in each student's test booklet, students were 
asked to complete two sections of back- 
ground questions regarding tlieir home or 
school experiences related to writing 
achievement. In total, the time required for 
each student to participate in the 2002 
NAEP writing assessment was no more 
than 1 hour. 
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School and Student Samples 

At the national level, results are reported 
for both public- and nonpublic-school 
students. At the state or jurisdiction level, 
results are reported only for public-school 
students. In order to obtain a representative 
sample of students for reporting national 
and state or jurisdiction results, approxi- 
mately 3,000 students from approximately 
100 schools per state or jurisdiction were 
sampled. In each state that did not partici- 
pate, a small number of students propor- 
tionate to the number of students in that 
state were sampled to contribute to the 
national sample. The total sampled for the 
2002 writing assessment included approxi- 
mately 139,200 fourth-grade students in 
5,500 schools; 118,500 eighth-graders in 
4,700 schools; and 18,500 twelfth-graders in 
700 schools. Each selected school and 
student participating in the assessment 
represents a portion of the total population. 
The administration procedures for the 2002 
assessment permitted testing accommoda- 
tions for students with disabilities and 
limited English proficient students who 
required them in order to participate. For 
information on sample si:^es and participa- 
tion rates by state or jurisdiction, see 
appendix A. 

Evaluating Students' Writing 
on the NAEP Assessment 

Student responses in the NAEP 2002 
writing assessment were evaluated according 
to scoring guide criteria describing six 
performance levels: Unsatisfactory, Insuffi- 
cient, Uneven, Sufficient, Skillful, and 
Excellent. Scoring guides were developed 
for narrative, informative, and persuasive 



writing at each grade level. A scale of 1 to 6 
representing these performance levels was 
used to evaluate each student response. The 
guides included specific notes for raters 
describing various student approaches to the 
task and offering anchor or prototypical 
student responses at each grade level. For 
each task, a wide spectrum of student 
approaches was judged acceptable. Ac- 
knowledging developmental differences 
between fourth-, eighth-, and twelfth- 
grades, the scoring guides (presented in 
chapter 6) reflect higher performance 
expectations for students in higher grades. 
Following the framework, the scoring guides 
emphasize students’ abilities to develop and 
elaborate ideas, organize tlieir thoughts, and 
write grammatically correct prose. The 
criteria for measuring command of written 
English mechanics differed by grade, but 
were the same across the three purposes for 
writing (narrative, informative, and persua- 
sive) within each grade. 

To give students an opportunity to plan 
their writing, NAEP provided a page for 
students to engage in such planning activi- 
ties as rough drafts, outlines, lists, diagrams, 
and pictures. Students, although not re- 
quired to plan their writing, were also given 
pamphlets with ideas about planning, 
editing, and revising writing and were 
encouraged to utilize them in the assess- 
ment. Recognizing that a time-controlled 
writing context constrains students’ oppor- 
tunities to plan and revise, responses to 
assessment tasks were viewed as first drafts 
and evaluated accordingly. (Further infor- 
mation about scoring is located in chapter 5.) 
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Reporting the Writing 
Assessment Results 

Results from the N^\EP writing assessment 
are presented in two ways: as scale scores 
and as percentages of students attaining 
achievement levels. The scale scores, indi- 
cating how much students know and can do in 
writing, are presented as average scale scores 
and as scale scores at selected percentiles. 
The achievement level results provide 
further information by indicating the degree 
to which student performance meets the 
standards set for what they should know and 
be able to do. Results are reported only for 
groups or subgroups of students; an indi- 
vidual student's performance cannot be 
reported based on NAEP assessment. 

Student responses to all tasks were 
analyzed to determine tlie percentage of 
students scoring at each level on the 6-level 
guides for narrative, informative, and 
persuasive writing. The analysis entails 
summarizing the results on separate 
subscales for each writing purpose and then 
combining the separate scales to form a 
single composite writing scale. This analysis 
yields the overall scale of 0 to 300 for each 
of tlie grades, 4, 8, and 12. Performance for 
each grade is scaled separately; therefore, 
average scale scores cannot be compared 
across grades. For example, equal scores on 
grade 4 and grade 8 scales do not imply 
equal levels of writing ability. (See the 
section on data analysis and IRT scaling in 
appendix A for more information on 
scaling procedures.) 



Achievement level results are presented in 
terms of writing achievement levels as 
authorized by NAEP legislation and 
adopted by NAGB. For each grade assessed, 
NAGB has adopted three achievement 
levels, BasiCy Profidenty Advanced. For 
reporting purposes, achievement level cut 
scores are placed on the writing scale to 
show the following ranges: below BasiCy 
BasiCy Proficienty and Advanced. The achieve- 
ment level results are then reported as 
percentages of students within each 
achievement level range, as well as the 
percentage of students at or above Basic '^nd 
at or above Proficient. 

The Setting of 
Achievement Levels 

The 1988 NAEP legislation that created die 
National Assessment Governing Board 
(NAGB) directed the Board to identify 
“appropriate achievement goals . . . for each 
subject area" that NAEP measures.'^ The 
2001 NAEP reauthorization reaffirmed 
many of the Board's statutory responsibili- 
ties, including developing “appropriate 
student achievement levels for each grade or 
age in each subject area to be tested . . . 

In order to follow this directive and achieve 
die mandate of the 1988 statute “to im- 
prove the form and use of NAEP results," 
NAGB undertook the development of 
student performance standards (called 
“achievement levels"). Since 1990, the 
Board has adopted achievement levels in 
mathematics, reading, US. history, geogra- 
phy, science, writing, and civics. 



Natioual Education Statistics Act. National Assessment of Educational Progress Improvement Act, Pub. L. No. 
100-297, 20 U.S.C. §1221 (1988). 

5 No Child Left Behmd Act of 2001, Pub. L. No. 107-110, 115 Stat. 1425 (2002). 
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The Board defined three achievement 
levels for each grade. The Basiclcvd denotes 
partial mastery of the knowledge and skills 
that arc fundamental for proficient work at 
a given grade. The Proficient \cvc\ represents 
solid academic performance. Students 
reaching this level demonstrate competency 
over challenging subject matter. The 
Adranced\cve\ presumes mastery of botli 
the Basic nnd Pwficient levels and represents 
superior performance. Figure 1.2 presents 
tlic policy definitions of the achievement 
levels that apply across grades and subject 
areas. The policy definitions guided the 
development of the writing achievement 
levels, as well as the achievement levels 
established in all other subject areas. 
Adopting three levels of achievement for 
each grade signals the importance of 
looking at more than one standard of 
performance. In the Board’s view, the 
overall achievement goal for American 
students is performance that qualifies at 
the Proficient level or higher as measured 
by NAEP. The Basic level is not the de- 
sired goal, but rather represents partial 
mastery that is a step toward Proficient. 



The achievement levels in this report 
were adopted by the Board based on a 
standard-setting process designed and 
conducted under a contract with ACT. To 
develop these levels, ACT convened a cross 
section of educators and interested citizens 
from across the nation and asked tliem to 
judge what students should know and be 
able to do relative to a body of content 
reflected in the NAEP assessment frame- 
work for writing. This achievement level 
setting process was reviewed by an array of 
individuals that included policymakers, 
representatives of professional organiza- 
tions, teachers, parents, and otlicr members 
of the general public. Prior to adopting 
these levels of student achievement, NAGB 
engaged a large number of individuals to 
comment on the recommended levels and 
to review the results. 

The results of the achievement level 
setting process, after NAGB’s approval, 
become a set of achievement level descrip- 
tions and a set of achievement level cut 
scores on the 0—300 NAEP writing scale. 
These levels are used to describe student 
performance on the 1998 and 2002 writing 
assessments. 



Figure 1.2 Policy definitions of the three NAEP achievement levels 



Achievement Levels 



Bask This level denotes partial mastery of prerequisite knowledge ond skills thot ore Fundamental For 

proFicient work ot eoch grode. 

Profkient This level represents solid ocodemic performonce for eoch grode ossessed. Students reaching this 

level have demonstroted competency over chollenging subject motter, including sufaject-motter 
knowledge, applicotion of such knowledge to reohworld situations, ond onolyticol skills appropriate 
to the subject motter. 

Advanced This level signifies superior performonce. 



SOURCE: Hationol Assessmcnl Governing Board. Wriling framework and Spetifkotiom for the 1998 national Assessment of fdarotlonol Progress. Worhinglon, DC: Auliror. 
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Writing Achievement Level 
Descriptions for Each Grade 

Specific definitions of the writing achieve- 
ment levels for grades 4, 8, and 12 are 
presented in figures 1.3 through 1.5. As 
noted previously, the achievement levels are 
cumulative. Therefore, students performing 
at the Proficient level also display the compe- 
tencies associated with the Basic level, and 
students at the Advanced level also demon- 



strate the competencies associated with 
both the Basic znd the Profiicient \cyc\s. For 
each achievement level listed in figures 1.3 
through 1.5, the scale score diat corre- 
sponds to die lowest cut score of that level 
on the NAEP writing scale is shown in 
parentheses. For example, in figure 1.3 the 
scale score of 176 corresponds to the lowest 
score of the grade 4 Proficient level of 
achievement in writing. 



Figure 1.3 Descriptions of NAEP writing achievement levels, grade 4 



Grade 4 

Achievement Levels 



The following stotements describe the kinds of things fourth-grode students should be oble to do in writing at eoch level of achievement. 
These statements should be interpreted with the constraints of the Notional Assessment of Educotionol Progress (NAEP) in mind. Student 
performonces reported with respect to these descriptions ore in response to two oge-oppropriote writing tosks completed within 25 minutes 
each. Students ore not advised of the writing tasks in odvonce nor engoged in pre-writing instruction and pfeporotion; however, they are 
given o set of "ideos for plonning ond reviewing" their writing for the ossessment. Although the Writing NAEP connot fully assess students' 
abilities to produce o polished piece of writing, the results do provide voluoble information about students' abilities to generate writing in 
response to o variety of purposes, tosks, and oudiences within a rather limited period of time. 



Basic Fourth-grade students performing at the Basic level should be able to produce a somewhat organized and detailed 

(115) response within the time allowed that shows a general grasp of the writing task they have been assigned. 

Fourth-grade students performing at the Has/rlevel should be able to produce a somewhat organized response within the time allowed 
that shows a general grasp of the writing task they have been assigned. Their writing should include some supporting details. Its 
grammar, spelling, and capitalization should be accurate enough to communicate to a reader, although there may be mistakes that get 
in the way of meaning. 



Proficient Fourth-grade students performing at the Proficient level should be able to produce an organized response within 
(176) allowed that shows an understanding of the writing task they have been assigned. Their writing should 

include details that support and develop their main idea, and it should show that these students are aware of the 
audience they are expected to address. 

Fourth-grade students performing at the Pro/iic/enr level should be able to produce an organized response within the time allowed that 
shows an understanding of the writing task they have been assigned. Their writing should include details that support and develop the 
main idea of the piece, and its form, content, and language should show that these students are aware of the audience they are 
expected to address. The grammar, spelling, and capitalization in the work should be accurate enough to communicate to a reader; 
there may be some mistakes, but these should not get in the way of meaning. 



Advanced Fourth-grade students performing at the Advanced level should be able to produce an effective, well developed 

(225) cesponse within the time allowed that shows a clear understanding of the writing task they have been assigned 

and the audience they are expected to address. Their writing should include details and be clearly organized, should 
use precise and varied language, and may show signs of analytical, evaluative, or creative thinking. 

Fourth-grade students performing at the Advanced level should be able ta produce an effective, well developed response within the 
time allowed that shows a clear understanding of the writing task they hove been assigned. Their writing should be clearly organized, 
making use of techniques such as consistency in topic ar theme, sequencing, and a clearly marked beginning and ending. It should 
make use of precise and varied language to speak to the audience the students are expected to address, and it should include details 
and elaboration that support and develop the main idea of the piece. Their writing may also show signs of analytical, evaluative, or 
creative thinking. The grammar, spelling, and capitalization in the work should be accurate enough to communicote clearly; mistakes 
should be so few and so minor that a reader can eosily skim over them. 

SOURCE: Naliond Assessmenl Governing Board. 12001 ). national Assessment of Educatiomd Progress Achievement levels, 1992-1998 for Writing. S.C Loomis and M.L Bourque (Eds.). 

Wodiington, DC: Author. 
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Figure 1.4 Descriptions of NAEP writing achievement levels, grade 8 



Grade 8 

Achievement Levels 



The following statements describe the kinds of things eighth>grade students should be able to do in writing at each level of achievement. 
These statements should be interpreted with the constraints of the National Assessment of Educational Progress (NAEP) in mind. Student 
performances reported with respect to these descriptions are in response to two age*appropriate writing tasks completed within 25 minutes 
each. Students are not advised of the writing tasks in advance nor engaged in pre>writing instruction and preparation; however, they are 
given a set of "'ideas for planning and reviewing" their writing for the assessment. Although the Writing NAEP cannot fully assess students" 
abilities to produce a polished piece of writing, the results do provide valuable information about students" abilities to generate writing in 
response to a variety of purposes, tasks, and audiences within a rather limited period of time. 



Basic Eighth-grade students performing at the Bask /eve/ shouid be able to produce an effective response within the time 

(] ] 4) aiiowed that shows a general understanding of the writing task they have been assigned. Their writing should 

show thot these students are awore of the audience they are expected to address, and it shouid include supporting 
detaiis in an organized way. 

Eighth-grade students performing at the fos/c level should be able to produce an effective response within the time allowed that shows 
a general understanding of the writing task they have been assigned. Their writing should show that these students are aware of the 
audience they are expected to address, and it should include supporting details in an organized way. The grammar, spelling, punctuation, 
and capitalization in the work should be accurate enough to communicate to a reader, although there may be mistakes that get in the 
way of meaning. 



Proficient Eighth-grade students performing at the Proficient level should be able to produce a detailed and organized response 

|] 73 j within the time allowed that shows an understanding of both the writing task they have been assigned and the 

audience they ore expected to address. Their writing should include precise language and varied sentence structure, 
and it may show analytical, evaluative, or creative thinking. 

Eighth-grade students performing at the Proficient level should be able to produce an effective response within the time allowed that 
shows an understanding of both the writing task they have been assigned and the audience they are expected to address. Their writing 
should be organized, making use of techniques such as sequencing or a clearly marked beginning and ending, and it should make use 
of details and some elaboration to support and develop the main idea of the piece. Their writing should Include precise language and 
some variety in sentence structure, and it may show analytical, evaluative, or creative thinking. The grammar, spelling, punctuation, 
and capitalization in the work should be accurate enough to communicate to a reader; there may be some errors, but these should not 
get in the way of meaning. 



Advanced Eighth-grade students performing at the Advanced level should be able to produce a fully developed response 
( 224 ) within the time allowed that sho ws a clear understanding of both the writing task they have been assigned and the 

audience they are expected to address. Their writing should show some analytical evaluative, or creative thinking 
and may make use of literary strategies to clarify a point. At the same time, the writing should be clearly organized, 
demonstrating precise word choice and varied sentence structure. 

Eighth-grade students performing at the Advancedkvel should be able to produce an effective and fully developed response within the 
time allowed that shows a clear understanding of both the writing task they have been assigned and the audience they are expected to 
address. Their writing should show some analytical, evaluative, or creative thinking, and should demonstrate precise word choice and 
varied sentence structure. Their work should include details and elaboration that support and develop the main idea of the piece, and 
it may make use of strategies such as analogies, illustrations, examples, anecdotes, or figurative language to clarify a point. At the 
same time, the writing should show that these students can keep their work clearly and consistently organized. Writing by eighth-grade 
students performing at the Advanced level should contain few errors in grammar, spelling, punctuation, capitalization, and sentence 
structure. These writers should demonstrate good control of these elements and may use them for stylistic effect in their work. 



SOURCE: Nalional Assessment Governing Board. (200) ). Nat’mai Asseamni of Educational Progress Achievemeni Levels, 1992-1998 for Writing. S.C Loomis and Ml Bourque (Eds.). 
Woshington, DC: Author. 
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Figure 1.5 Descriptions of NAEP writing achievement levels, grade 12 



Grade 1 2 
Achievement Levels 



The fallowing statements describe the kinds af things twelfth-grade students should be able ta do in writing at each level of achievement. 
These statements should be interpreted with the constraints of the National Assessment of Educational Progress (NAEP) in mind. Student 
performances reported with respect to these descriptions are in response to two age-appropriate writing tasks completed within 25 minutes 
each. Students are nat advised of the writing tasks in advance nar engaged in pre-writing instruction and preparation; however, they are 
given a set of "ideas for planning and reviewing" their writing for the assessment. Although the Writing NAEP cannot fully assess students' 
abilities to produce a polished piece of writing, the results do provide valuable information about students' abilities to generate writing in 
response to a variety of purposes, tasks, and audiences within a rather limited period of time. 



Basic Twelfth-grade students performing at the Basic level should be able to produce a well-organized response within 
(1 22) allowed that shows an understanding of both the writing task they have been assigned and the audience 

they are expected to address. Their writing should show some analytical, evaluative, or creative thinking, and it 
should include details that support and develop the main idea of the piece. 

Twelfth-grade students performing at the tor level should be able to produce an effective response within the time allowed that shows 
an understanding of both the writing task they have been assigned and the audience they are expected to address. Their writing should 
show some analytical, evaluative, or creative thinking. It should Include details that support and develop the central idea of the piece, 
and it should be clearly organized, making use of techniques such as consistency in topic or theme, sequencing, and a clear introduction 
and conclusion. The grammar, spelling, punctuation, and capitalization in these students' work should be accurate enough to communicate 
to a reader; there may be some errors, but these should not get in the way of meaning. 

Proficient Twelfth-grade students performing at the Profident level should be able to produce an effectively organized and 

(178) felly developed response within the time allowed that uses analytical, evaluative, or creative thinking. Their 

writing should indude details that support and develop the main idea of the piece, and It should show that these 
students are able to use precise language and variety in sentence structure to engage the audience they are 
expected to address. 

Twelfth-grade students performing at the Profkientlml should be able to produce an effective and fully developed response within 
the time allowed that uses analytical, evaluative, or creative thinking. Their writing should be organized effectively, and it should show 
that these students have a clear understanding of the writing task they have been assigned. It should be coherent, making use of 
techniques such as a consistent theme, sequencing, and a clear introduction and conclusion, and it should include details and elaboration 
that support and develop the main idea of the piece. The writing should show that these students are able to use precise language and 
variety in sentence structure to engage the audience they are expected to address. Writing by twelfth-grade students performing at the 
/’rofoent level should contain few errors in grammar, spelling, punctuation, capitalization, and sentence structure. These writers should 
demonstrate a command of these elements and may use them for stylistic effect in their work. 



Advanced Twelfth-grade students performing at the Advanced level should be able to produce a mature and sophisticated 
(230) response within the time allowed that uses analytical evaluative, or creative thinking. Their writing should be 
detailed and fully developed, and it should show that these students ore able to use literary strategies to develop 
their ideas. At the same time, the writing should be well crafted and coherent, and it should show that these 
students are able to engage the audience they are expected to address through rich and compelling language, 
precise word choice, and variety in sentence structure. 

Twelfth-grade students performing at the Advanced level should be able to produce a mature and sophisticated response within the 
time allowed that uses analytical, evaluative, or creative thinking. Itieir writing should be fully developed, incorporating details and 
elaboration that support and extend the main idea of the piece. It should show that these students can use literary strategies— 
anecdotes and repetition, for example— to develop their ideas. At the same time, the writing should be well crafted, organized, and 
coherent, and it should incorporate techniques such as a consistency in topic or theme, sequencing, and a clear introduction and 
conclusion. It should show that these writers can engage the audience they are expected to address through rich and compelling 
language, precise word choice, and variety in sentence structure. Writing by twelfth-grade students performing at the Advanced level 
should contain few errors in grammar, spelling, punctuation, capitalization, and sentence structure. These writers should demonstrote 
a sophisticated command of these elements and may use them for stylistic effect in their work. 



SOURCE; National Assessment Governing Board. (2001 ). National Assessment of Educational ^ogress Achievement levels, 1 992”I998 for Writing. S.C. Loomis and M.L Bourque (Eds.). 
Woshinglon, DC: Author. 
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Trial Status of 
Achievement Levels 

As provided by law and based upon a 
review of congressionally mandated evalua- 
tion of NAEP, NCES has determined that 
achievement levels are to be used on a trial 
basis and should be used with caution until 
NCES determines their.validity. In 1993, the 
first of several congressionally mandated 
evaluations of the achievement level setting 
process concluded that the procedures used 
to set die achievement levels were flawed 
and that the percentage of students at or 
above any particular achievement level cut 
point may be underestimated.^ Others have 
critiqued these evaluations, asserting that the 
weight of the empirical evidence does not 
support such conclusions.^ 

In response to die evaluations and 
critiques, NAGB conducted an additional 
study of die 1992 reading achievement 
levels before deciding to use them for 
reporting 1994 NAEP results.® When 
reviewing the findings of this study, the 
National Academy of Education (NAE) 
panel expressed concern about what it saw 



as a ‘‘confirmatory bias” in the study and 
about the inability of this study to “address 
the panel’s perception that the levels had 
been set too high.”^ In 1997, the NAE panel 
summarized its concerns about interpreta- 
tion of NAEP results based on die achieve- 
ment levels as follows: 

First, the potential instability of the 
levels may interfere with die accurate 
portrayal of trends. Second, the percep- 
tion that few American students are 
attaining the higher standards we have 
set for them may deflect attention to die 
wrong aspects of education reform. The 
public has indicated its interest in 
benchmarking against international 
standards, yet it is noteworthy that when 
American students performed very well 
on a 1991 international reading assess- 
ment, these results were discounted 
because they were contradicted by poor 
performance against the possibly flawed 
N^\EP reading achievement levels in die 
following year.^^ 



^ United States General Accounting Office. (1993). Hducation A.chievtment Standards: ISlAG^s ^Approach Yields Misleading 
Interpretations, US. General Accmmting Office Report to Congressional Requestors. Washington, DC: Author. 

National Academy of Education. (1993). SettingPerformance Standards for Achievement: A Report of the National Academy 
of Education Panel on the Evaluations of the NAEP Trial State Assessment: An Evaluation of the 1 992 Achievement Ijevels. 
Stanford, CA: Author. 

^ Cizek, G. (1993). Reactions to National Academy of Education Report. Wasliington, DC: National Assessment Governing 
Board. 

Kane, M. (1993). Comments on the NAEP Evaluation of the NAGB Achievement Levels. Wasliington, DC: National 
Assessment Governing Board. 

® American College Testing. (1995). NAEP Reading Revisited: An Evaluation of the 1992 Achievement Level Descriptions. 

Washington, DC: National Assessment Governing Board. 

^ National Academy of Education. (1996). Reading Acliicvcmcnt Levels. \w Quality and Utility: The 1994 Trial State 
Assessment in Reading. The Fourth Report of the National Acaderty of Education Panel on the Evaluation of the NAEP Trial 
State Assessment. Stanford, CA: Author. 

National Academy of Education. (1997). Assessment in Transition: Monitoring the Nation^s Educational Progress^ p. 99. 
Momitain View, CA: Author. 
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NCES and NAGB continue to seek new 
and better ways to set performance stan- 
dards on NAEP.“ For example, NCES and 
NAGB jointly sponsored a national confer- 
ence on standard setting in large-scale 
assessments, which explored many issues 
related to standard setting. Although new 
directions were presented and discussed, a 
proven alternative to the current process has 
not yet been identified. NCES and NAGB 
continue to call on the research community 
to assist in finding ways to improve standard 
setting for reporting NAEP results. 

The most recent congress ionally man- 
dated evaluation, conducted by the National 
Academy of Sciences (NAS), relied on prior 
studies of achievement levels, rather than 
carrying out new evaluations, on the 
grounds that the process has not changed 
substantially since the initial problems were 
identified. Instead, the NAS panel studied 
the development of the 1996 science 
achievement levels. The NAS panel basically 
concurred with earlier congressionally 
mandated studies. The panel concluded that 
“NAEP’s current achievement level setting 
procedures remain fundamentally flawed. 
The judgment tasks are difficult and confus- 
ing; raters’ judgments of different item 
types are internally inconsistent; appropriate 



validity evidence for the cut scores is lack- 
ing; and die process has produced unreason- 
able results.” 

The NAS panel accepted the continuing 
use of achievement levels in reporting 
NAEP results on a trial basis, until such 
time as better procedures can be developed. 
Specifically, the NAS panel concluded that 

. . tracking changes in die percentages of 
students performing at or above those cut 
scores (or in fact, any selected cut scores) 
can be of use in describing changes in 
student performance over time.”^"^ 

NAGB urges all who are concerned 
about student performance levels to recog- 
nize that the use of these achievement levels 
is a developing process and is subject to 
various interpretations. NAGB and NCES 
believe that the achievement levels are 
useful for reporting trends in the educa- 
tional achievement of students in the 
United States.^^ In fact, achievement level 
results have been used in reports by the 
President of the United States, the Secretary 
of Education, state governors, legislators, 
and members of Congress. Government 
leaders in the nation and in more than 40 
states use these results in their annual 
reports. 



Reckase, M. D. (2000). The Evolution of the NARP y\chievement Ijevel Setting Process, Summary of the Research and 

Development of Rfforts Conducted by ACT. Iowa City, lA: ACT, Inc. 

National Assessment Governing Board and National Center for Education Statistics. (1995). Proceedings of the Joint 
Conference on Standard Setting for luirge-S cale Assessments of the Plational Assessment Governing Board (ISJAGB) and the 
National Center for Rducation Statistics (NCES). Wasliington, DC; Govenunent Printing Office. 

Pellegrino, J. W, Jones, L. R., and Mitchell, K. J. (Eds.). (1999). Grading the Nation*s Keport Card: Evaluating NAEP and 
Transforming d)e Assessment of Educational Progress. Committee on the Evaluation of National and State Assessments of 
Educational Progress, Board on Testing and Assessment, Commission on Behavioral Social Sciences and Education, 
National Research Council. Washington, DC: National Academy Press. 

Ibid., 176. 

Forsydi, R. A. (2000). A Description of die Standard-Setting Procedures Used By lliree Standardi 2 ed Test Publish- 
ers. In M. L. Bourque, (Ed.), Student Performance Standards on the National Asse.ssment of Educational Progress: Affirmations 
and Improvements. Washington, DC: National Assessment Governing Board. Available http:/ / www.nagb.org/piibs 
NelUiaus, J. M. (2000). States widi NAEP-Like Petfotniance Standards. In M. L. Bourque, (Ed.), Student Performance 
Standards on the National Assessment of Educational Progress: Affirmations and Improvements. Wasliington, DC: National 
Assessment Governing Board. 
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However, based on the congressionally 
mandated evaluations so far, NCES agrees 
with the NAS panel’s recommendation that 
caution needs to be exercised in the use of 
die current achievement levels. Therefore, 
NCES concludes that these achievement 
levels should continue to be used on a trial 
basis and should continue to be interpreted 
and used with caution. 

Interpreting NAEP Results 

The average scores and percentages pre- 
sented in this report are estimates based on 
samples of students rather than on entire 
populations. Moreover, the collection of 
questions used at each grade level is but a 
sample of the many questions that could 
have been asked to assess the skills and 
abilities described in the NAEP writing 
framework. As such, the results are subject 
to a measure of uncertainty, reflected in die 
standard error of the estimates — a range of 
a few points plus or minus the score or 
percentage — which accounts for potential 
score or percentage fluctuation due to 
sampling and measurement error. The 
estimated standard errors for the estimated 
scale scores and percentages in this report 
are accessible through the NAEP Data Tool 
on the NAEP web site (http://nces.ed.gov/ 
nationsreportcard/naepdata/). Examples of 
these estimated standard errors are also 
provided in appendix A, tables A.8 to A. 12, 
of this report. 



The differences between scale scores and 
between percentages discussed in the 
following chapters take into account the 
standard errors associated with the esti- 
mates. Comparisons are based on statistical 
tests that consider both the magnitude of 
the difference between the group average 
scores or percentages and tlie standard 
errors of those statistics. Estimates based 
on smaller subgroups are likely to have 
relatively large standard errors. As a conse- 
quence, some seemingly large differences 
may not be statistically significant. That is, it 
cannot be determined whether these differ- 
ences are due to tlie particular makeup of 
the samples of students who. are selected, or 
to true differences in the population of 
interest. When this is the case, the term 
‘‘apparent difference” is used in this report. 
Differences between scores or between 
percentages are discussed in this report only 
when they are significant from a statistical 
perspective. All differences reported are 
significant at the .05 level (with appropriate 
adjustments for comparison between 
multiple subgroups of students). The term 
“significant” is intended to identify statisti- 
cally dependable differences in average 
scores or percentages and not to imply a 
judgment about the absolute magnitude or 
the educational relevance of the differences. 
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Readers are cautioned against interpreting 
NAEP results in a causal sense. Inferences 
related to subgroup performance or to the 
effectiveness of public and nonpublic 
schools, for example, should take into 
consideration the many socioeconomic and 
educational factors that may affect writing 
performance. 

Overview of the 
Remaining Report 

This report describes the writing perfor- 
mance of fourth-, eighth-, and twelfth- 
graders in the nation, as well as fourth- and 
eighth-graders in participating states and 
other jurisdictions. Chapter 2 presents 
overall writing scale scores and achievement 
level results across years for both the nation 
and participating states and other jurisdic- 
tions. Chapter 3 discusses national results 
for subgroups of students by gender, race/ 
ethnicity, parents’ highest level of education 
(for grades 8 and 12 only), type of school 
(public and nonpublic), type of school 
location (central city, urban fringe/large 
town, rural/small town). Title I participa- 
tion, and eligibility for free/reduced-price 
school lunch. State and jurisdiction results 
are reported by gender, race/ethnicity, and 
eligibility for free/reduced-price school 
lunch only. 



Chapter 4 presents sample writing tasks 
and sample student responses representing 
varying score levels at each grade. In addi- 
tion, item maps for each grade level describe 
the skill needed to respond to particular 
writing tasks and show the score points at 
which individual students had a high prob- 
ability of successfully writing in response to 
particular tasks, thereby indicating the 
relative difficulty of each task. 

The appendices of tliis report contain 
information to expand the results presented 
in chapters 2-4. Appendix A contains an 
overview of assessment development, 
sampling, inclusion of special-needs stu- 
dents and use of accommodations, adminis- 
tration, and analysis procedures. Appendix 
B presents the percentages of students in 
each of the subgroups reported for the 
nation and states or other jurisdictions. 
Finally, appendix C shows state-level con- 
textual data from sources other tlian NAEP. 
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Average Writing Scale Score 
and Achievement Level Results 
for the Nation and States 
Overview 

This chapter presents the NAEP 2002 writing results for 
public- and nonpublic-school students in the nation at 
grades 4, 8, and 12 and for public-school students in 
participating states and jurisdictions at grades 4 and 8. 
Average scores on the NAEP writing composite scale 
range from 0 to 300; the three writing achievement levels 
are Basic, Proficient, and Advanced. 

In addition to the results from the 2002 writing 
assessment, results are presented from 1998 for the nation 
at aU three grades and for participating states and other 
jurisdictions at grade 8. There was no state -level assessment 
at grade 4 in 1998. At grades 4 and 8, the national sample 
in 2002 was a subset of the combined sample of students 
assessed in each participating state plus an additional 
sample from the states that did not participate in the state 
assessment. Although results were presented by region of 
the country (Northeast, South, Central, and West) in 
previous reports, regional data are not presented in this 
year’s report because low participation in some states that 
did not participate in the state assessment made the 
comparative data for two of the regions less reliable than 
in the past 

National Writing Scale Score Results 

Figure 2.1 displays the average writing scores from 1998 
and 2002 for fourth-, eighth-, and twelfth-graders. Results 
for each grade are scaled independently; therefore, cross- 
grade score comparisons cannot be made. Students’ 
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average scores on the NAEP writing 
assessment increased between 1998 and 
2002 at grades 4 and 8. However, there 



was no significant change detected in the 
performance of twelfth-graders between 
the two assessment years. 



Figure 2.1 Average writing scale scores, grades 4, 8, and 1 2: 1998 and 2002 



Grades 4, 8, and 1 2 



Grade 4 Grade 8 Grade 1 2 






* Signjfkontly diffexen! from 2002. 

SOURCE U5. Depoilmenl of Education, Institute of Education Sdeiros, Notional Center for Education Statistics, National AsessmenI of Educational Pnqress (KAff), 1 998 and 2002 Writing Assessments. 



National Writing Scale Scores 
by Percentile 

Another way to view students' perfor- 
mance is by looking at how scores have 
changed across the performance distribu- 
tion. An examination of scores at different 
percentiles on the 0—300 writing scale at 
each grade indicates whether or not the 
changes seen in the overall national average 
score results are reflected in the perfor- 
mance of lower-, middle-, and higher- 
performing students. Results for each grade 
are scaled independently; therefore, cross- 
grade score comparisons cannot be made. 
Figure 2.2 shows the average writing scale 
score for students scoring at the 10th, 25th, 
50th, 75th, and 90th percentiles at all three 



grade levels. The percentile indicates the 
percentage of students whose scores fell 
below a particular point on the NAEP 
writing scale. For example, tlie 75tli percen- 
tile score at grade 4 was 179 in 2002, 
indicating that 75 percent of fourth-graders 
scored below 179. 

Increases in fourtli-grade writing scores 
were observed across the distribution. 

Gains were observed among the middle- 
and higher-performing students at grade 8; 
no significant changes were detected at the 
10th and 25th percentiles. At grade 12, 
only the score at the 90th percentile in- 
creased since 1998, while scores at the 
10th and 25th percentiles were lower in 
2002. 
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Figure 2.2 Writing scale score percentiles, grades 4, 8, and 1 2: 1998 and 2002 



Grades 4, 8, and 1 2 
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Grade 8 



Grade 1 2 
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* SignificDitfly different from 2002. 

SOURCE: Ui. Departm^t of Education, Institute of Education Sdences., National Center for Educotion Statistics, National Assessment of Educational Progress (NMP), 1998 and 2002 Writing Assessments. 



National Writing Achievement 
Level Results 

In addition to reporting average writing 
scale scores, NAEP reports writing perfor- 
mance by achievement levels. The writing 
achievement levels are Bask\ Proficient, and 
Advanced. Discussion related to the setting 
of achievement levels is covered in chapter 1 
Figure 2.3 tracks the percentages of 
students performing at or above Basic and 
at or above Proficient — the level identified 
by NAGB as the level at which all students 
should perform — across assessment years. 



Table 2.1 presents the achievement level 
results in two ways for each grade: as the 
percentage of students within each 
achievement level, and as the percentage 
of students at or above the Basic level and 
at or above the Proficient level. The percent- 
ages at or above specific achievement 
levels are cumulative. Included among the 
percentage of students at or above the Basic 
level are those who have achieved the 
Proficient and Advanced levels of perfor- 
mance. Included among students at or 
above the Proficient level are those who 
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have attained the Advanced level of perfor- 
mance. Although significant differences in 
the percentages of students within achieve- 
ment levels are indicated in the table, only 
die differences at or above BasiCy at or 
above Pwficient, and at Advanced are dis- 
cussed in this section. 

In 2002, between 24 and 31 percent of 
die students in each of the three grades 
performed at or above the P?vjicient level. 
Figure 2.3 shows that fourth- and eighth- 
graders have made overall gains since 1998 
in reaching the Proficient level, while there 
was no significant change detected in the 
percentage of twelfth-graders at or above 
this level over the same period of time. 

As shown in more detail in table 2.1, 
trends in achievement level results are 



generally consistent with trends in average 
scale score results since 1998 that are 
described in the previous section. The 
percentages of fourth-graders at or above 
Basic and at or above Proficient increased 
over the period between the 1998 and 2002 
assessments. No significant change was 
observed in the percentage of eighdi- 
graders performing at or above Basic, while 
the percentage of eighth-grade students 
performing at or above Proficient increased 
over the same interval. The percentage of 
twelfth-graders performing at or above 
Basic decreased since 1998. Although only 
2 percent of the students in each grade 
performed at tlie Advanced level in 2002, 
this did reflect an increase over the per- 
centages in 1998. 



Figure 2.3 Percentage of students at or obove Bas/cand Proficient \n writing/ grades 4, 8, and 12: 1998 and 2002 



Grades 4, 8, and 1 2 




Grade 12 



IJJIL 

ilL 

liL 

ilL 

ilL 



78 



-74- 



iO- 

JLD_ 

0 



•98 '02 



Percent at or 
above Basic 

Percent at or 
above Proficient 



* Si^konlly diHerent from 2001 

SOURCE: WS. Depoilmenl of Education, Institute of Education Sciences, Notional Center for Education Slathtio, Nationd Assessment of Educotiond Progress (KAEP), 1998 and 2002 Writing Assessmei^ 
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Table 2.1 Percentage of students, by writing achievement level, grades 4, 8, and 1 2: 1 998 and 2002 
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At or above 


: At;dr;above " 




Below Bask 


At Bask 


At Profkient 


At Advanced 


Bask 


















1998 


16* 


61* 


22* 


1* 


84* 


23* 


2002 


14 


58 


26 


2 


86 


28 


Grade 














1998 


16 


58* 


25* 


1* 


84 


27* 


2002 


15 


54 


29 


2 


85 


31 
















1998 


22* 


57* 


21 


1* 


78* 


22 


2002 


26 


51 


22 


2 


74 


24 



* Significantly different from 2002. 

KOTL Penentoges within eodi writing odiievement level range may not odd to 1 00, or to the exact percentoges at or above achievement levels, due to roun^g. 

SOURCE: U.S. Oepartmenl of Eihjcalion, Institute of Eduoilion Science^ National Center for Eduration Statistics, Notional Assessment of Educational Progress (NAEP), 1 998 and 2002 Writing Assessments. 



Writing Results for States and 
Other Jurisdictions 

In addition to the national results, writing 
performance data were collected for fourth- 
and eighth-grade students attending public 
schools in states and other jurisdictions 
that chose to participate in the 2002 
assessment.' Although 50 jurisdictions 
participated in the 2002 writing assessment 
at grade 4, and 50 participated at grade 8, 
not all met minimum school participation 
guidelines for reporting their results. (See 
appendix A for details on participation and 
reporting guidelines.) Results from the 2002 
assessment are not included for Illinois and 
Wisconsin at grades 4 and 8, and for 
Minnesota at grade 8, because they did not 
meet the minimum weighted school partici- 
pation rate of 70 percent. Jurisdictions that 
did not meet one or more of the other 
participation guidelines are noted in each 



of the tables. Information about students’ 
writing achievement in selected urban 
school districts (Atlanta, Chicago, Houston, 
Los Angeles, and New York City) is avail- 
able in the forthcoming NAEP Trial Urban 
District Assessment Writing 2002. 

Results are presented for jurisdictions 
that participated in the 2002 writing 
assessment at grade 4, and in the 1998 and 
2002 assessments at grade 8. Tables pre- 
senting state level results at grade 8 indi- 
cate statistically significant changes across 
years when examining only one jurisdiction 
at a time (*), or when using a multiple 
comparison procedure based on all the 
jurisdictions that participated (**). Differ- 
ences discussed in this report are based on 
statistically significant findings detected 
using either comparison procedure. (See 
appendix A for a more detailed discussion 
of comparison procedures.) 



Tliroiighout this chapter die term jurisdiction is used to refer to the states, territories, and Department of Defense 
schools that participated in tlie NAEP writing assessments. 
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Writing Scale Score Results 
by State/Jurisdiction 

Average writing scale scores by jurisdiction 
are shown in table 2.2 for grade 4, and 
table 2.3 for grade 8. Whereas the national 
results presented in the previous sections 
of this chapter represent both public and 
nonpublic schools combined, the national 
average scores shown in each of these 



tables represent tlie performance of public- 
school students only. 

Average fourth-grade scores ranged from 
125 to 174. Of the 36 jurisdictions that 
participated in both the 1998 and 2002 
eighth-grade writing assessments, 16 
showed score increases in 2002, and none 
showed a significant decrease. 
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Toble 2.2 Averoge writing scole scores, grade 4 public schools: By stote, 2002 





2002 


Notion (Public) 


153 


Alabama 


140 


Arizona 


140 


Arkansas 


145 


California ^ 


146 


Connecticut 


174 


Delaware 


163 


Florida 


158 


Georgia 


149 


Hawaii 


149 


Idaho 


150 


Indiana 


154 


Iowa ^ 


155 


Kansas ^ 


149 


Kentucky 


154 


Louisiana 


142 


Maine 


158 


Maryland 


157 


Massachusetts 


170 


Michigan 


147 


Minnesota ^ 


156 


Mississippi 


141 


Missouri 


151 


Montana^ 


149 


Nebraska 


154 


Nevada 


145 


New Mexico 


142 


New York ^ 


163 


North Carolina 


159 


North Dakota^ 


150 


Ohio 


157 


Oklahoma 


142 


Oregon 


149 


Pennsylvania 


156 


Rhode Island 


157 


South Carolina 


145 


Tennessee ^ 


149 


Texas 


154 


Utah 


145 


Vermont 


158 


Virginia 


157 


Washington ^ 


158 


West Virginia 


147 


Wyoming 


150 


Other Junsdkttuns 




District of Columbia 


135 


DDESS > 


156 


DoDDS^ 


159 


Guam 


131 


Virgin Islands 


125 



^ Indicntes that the jurisdktion (fid not meet one or more of the guidefines for sdtod portii^ion m 2002. 



SOURCE: Ui. Deportment of Edurotion, tnsfriule of Edutofion Sdences, Notioral Center for Education Statctio, Notional Asessment of Etfixational Progress (NAEP), 2002 Writmg Assessment. 
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Table 2.3 Average writing scale scores, grade 8 public schools: By state, 1998 and 2002 





1998 


2002 


Nation (Public) ^ 


148* * 


152 


Alabama 


144 


142 


Arizona 


143 


141 


Arkansas 


137 *'** 


142 


Carifornia ^ 


141 


144 


Colorado 


151 




Connecticut 


165 


164 


Deloware 


144*'** 


159 


Florida 


142*'** 


154 


Georgia 


146 


147 


Howaii 


135 


138 


Idaho 


— 


151 


Indiana 


— 


150 


Kansas ^ 




155 


Kentucky 


146 


149 


Louisiana 


136 *'** 


142 


Maine 


155 


157 


Maryland 


147 *'** 


157 


Massachusetts 


155 *'** 


163 


Michigon 




147 


Minnesota ^ 


148 


— 


Mississippi 


134*'** 


141 


Missouri 


142 *'** 


151 


Montana ^ 


150 


152 


Nebraska 


— 


156 


Nevada 


140 


137 


New Mexico 


141 


140 


New York ^ 


146*'** 


151 


North Carolina 


150*'** 


157 


North Dakota^ 


— 


147 


Ohio 


— 


160 


Oklahoma 


152 


150 


Oregon ^ 


149* 


155 


Pennsylvania 


— 


154 


Rhode Island 


148*'** 


151 


South Carolina 


140*'** 


146 


Tennessee ^ 


148 


148 


Texas 


154 


152 


Utah 


143 


143 


Vermont 


— 


163 


Virginia 


153 


157 


Washington ^ 


148 *'** 


155 


West Virginia 


144 


144 


Wisconsin ^ 


153 


— 


Wyoming 
Other Jurisdictions 


146 *'** 


151 


American Samoa 


— 


95 


District of Columbia 


126 


128 


ODESSA 


160 


164 


DoDDS 3 


156*'** 


161 


Guam 


— 


130 


Virgin Islands 


124 


128 



— tndkates ihol the jurisdktion ihd not portidpote or ihd not meet miimum partidpatkm gutdetmes for reputing, 
t Imhcotes that the jurrsdiction not meet one or more of the guideTmes for school portic^ation m 2002. 

* Signihconlly different from 2002 when only one {urischction or the notion is being examined. 

** Signihomtly cfifferent from 2002 whm rang a multipie-compDrison procedure hosed on cdl ^msdkfions llud pcvtinpoted buh yeuv. 

^ Notioncdresdtsforthe 1 998 assessment ore b^d on in noionolscmipfe, not on crggregotedshile assessment sornples. 

^ Department of Defense Domestk Dependent Qementaiy Old Secondoy Schools. 

^ Deportmenl of Defense D^iendenls Schools (Overseas). 

NOIL Comparative performance results may be affected by choi^ in exclusion rates for students widi dscMties ami liirated Bighsh proficient students in the NA£P sanqdes. 

SOURCE: Department of Educotioo, Institute of Echjcotion Sciences, Hationd Center for Echjcatlon Statistics, Notional Assessment of Educational Progress (NAEP), 1998 cmd 2002 Writing Assessments. 
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The maps in figures 2.4 and 2.5 compare 
state and national average writing scores in 
2002 at grades 4 and 8 respectively. At 
grade 4, 17 jurisdictions had scores that 
were higher than the national average 
scores, 22 had scores that were lower than 
the national average, and no significant 
differences were detected between the 



jurisdiction and national averages for 9 
jurisdictions. At grade 8, 12 jurisdictions 
had scores that were higher than the 
national average scores, 20 had scores that 
were lower than the national average, and 
no significant differences were detected 
between the state and national average for 
15 jurisdictions. 



Figure 2.4 Comparison of state and national public school average writing scale scores, grade 4: 2002 



Gradr4 





Jurisdiction hod higher overoge scale score than nation. 

Jurisdiction was not found to be significantly different from nation in overage scale score. 
Jurisdiction had lower average scale score thon notion. 

Jurisdiction did not meet minimum participation rate guideDnes. 

Jurisdiction did not parlic^ate b the NAEP 2002 Writing State Assessment. 



f Depiitmenl of Defense Domes& Dependent Qemento^ 

^ Depolment of Defense DepoidenlsSdiools (Ovenem). 

SOURCE: Ui. Depaitnen! ^ Bfucotion, Institute of Educotion Sciences, Notiond Center for Eihicalion Stalis&s, Notional Assessment of Edixotkmd Progress (HAEP), 2002 WrUing Assessment. 
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Figure 2.5 Comparison of sfote and national public school average writing scale scores, grade 8: 2002 



Grade 8 




Jurisdicllon hqd higher overoge scole score than notion. 

Jurisdiction wos not found to be significontly different from notion in overage scale score. 
Jurisdiction hod lower overage scole score than nation. 

Jurisdiction did not meet minimum portkipotion rate guidelines. 

Jurisdiction did not portk^ote In the NAEP 2002 Writing State Assessment. 



^ Deportment of Defense Domestic Dependent Qementory and Secondoy Sdiooh. 

^ Deportment of D^ense D^iendents Sdiooh (Overseas). 

SOURCE: Ui. Department of Education institute of Education Sdencesi, National Center br Education Statistics, Notiomd Assessment of Educationol Progress (NAEP), 2002 VAritIng Assessment. 



Cross-State/Jurisdiction 
Writing Scale Score 
Comparisons 

Figures 2.6 and 2.7 display the differences 
in the N^\EP 2002 average writing scale 
scores between any two participating 
jurisdictions at grades 4 and 8 respectively. 
Tliese figures are set up similarly to mileage 
charts on travel maps. On the line across 
the top of the figure, find the name of the 
target jurisdiction and follow the column 
below the target jurisdiction to the jurisdic- 
tion chosen for comparison. If the cell of 



the comparison jurisdiction is not shaded, 
no statistically significant difference be- 
tween the scale scores of the two jurisdic- 
tions was detected. If the cell of the 
comparison jurisdiction is lightly shaded, 
the average scale score of that jurisdiction 
was higher than the scale score of the 
target jurisdiction named at the top of the 
column. Darkly shaded cells indicate that 
the average scale score of the comparison 
jurisdiction was lower dian that of the 
target jurisdiction selected at die top of the 
column. 
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At grade 4, Connecticut, Massachusetts, 
and Delaware were among the highest 
performing jurisdictions. The average 
writing score in Connecticut was higher 
than in any of the other participating 
jurisdictions. Massachusetts was outper- 
formed only by Connecticut. Students in 
Delaware were outperformed only by 
students in Connecticut and Massachusetts 
and had higher scores than the other 
participating jurisdictions except New York. 



At grade 8, Connecticut, Department of 
Defense domestic and overseas schools, 
Massachusetts, and Vermont were among 
the highest performing jurisdictions, and 
were not found to differ significantly from 
each otlier and had higher scores than the 
other participating jurisdictions except Ohio. 
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Figure 2.6 Cross-state comparison of average writing scole scores, grade 4 public schools: 2002 



, , , 




Instructions: Read down the column directly under a jurisdict 


on name listed in the heading at the top of the chart. Match the | 



of this jurisdiction was found to be higher than, not significantly different from, or lower than the jurisdiction in the column heading. For 
example, note the column under North Carolina: North Carolina's score was lower than Connecticut, Massachusetts, and 
Delaware, about the same as all the states from New York through Iowa, and higher than the remaining states down the column. 
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1 Jurisdiction hod higher overage scale score than the 
jurisdiction listed at the top of the figure. 



] Jurisdiction hod lower average scale score than the jurisdiction 
listed at the top of the figure. 



I No significant difference detected from the jurisdiction listed 
at the top of the figure. 



I Indicates that the jurisdiction did not meet one or more of the guidelines for school participation in 2002. 
^ Departmentof Defense Dependents Schools (Overseas). 

^ Department of Defense Domestic Dependent Elementary and Secondary Schools. 

NOTE: The between-jurisdiction comparisons take into account sampling ond measurement error and that 
each jurisdiction is being compared with every other jurisdiction. Significance h determined by an 
application of a multiple<omparison procedure (see appendix A). 

SOURCE U.S. Deportment of Education, Institute of Education Sciences, National Center for Education 
Statistics, Notional Assessment of Educational Progress (NAEP), 2002 Writing Assessment. 
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Figure 2.7 Cross-state comparison of overage writing scale scores, grade 8 public schools: 2002 



ERIC 






Instructions: ReocJ down the column directly under o jurisdiction nome listed in the heoding ot the top of the chort. Motch the 
shoding intensity surrounding o jurisdiction's obbreviotion to the Icey below to determine whether the overoge writing scole score 
of this jurisdiction wos found to be higher thon, not significontly different from, or lower thon the jurisdiction in the column heoding. 
For exomple, note the column under Virginio: Virginio's score wos lower thon Connecticut, Deportment of Defense Domestic 
Schools, Mossochusetts, Vermont, ond Deportment of Defense Overseos Schools, obout the some os oil the states from Ohio 
through Florido, ond higher thon the remoining states down the column. ^ 
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I I Jurisdiction hod higher overage scale score thon the 
jurisdiction listed at the top of the figure. 

I I No significont difference detected from the jurisdiction fisted 
at the top of the figure. 

I I Jurisdiction hod lower overoge scole score thon the jurisdiction 
listed at the top of the figure. 



T indicotes thot the junsdiction did not meet one or more ot the guidelines tor school 
^ Deportment of Defense Domestic Dependent Elementory ond Secondory Schools. 



NOTE The between-jurisdiction comporisons take into occount sampling ond measurement error ond that 
eoch jurisdiction is being compored with every other jurisdiction. Significance is determined by on 
opplicotion of o muitiple^omparison procedure (see oppendix A). 

SOURCE U.S. Deportment of Education, Institute of Education Sciences, Notiono! Center for Education 
Statistics, Notionol Assessment of Educationol Progress (NAEP), 2002 Writing Assessment. 
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Writing Achievement Level 
Results by State/Jurisdiction 

Achievement level results for jurisdictions 
are presented both as die percentage of 
students scoring within each writing 
achievement level range and as the percent- 
age of students performing at or above the 
Prvfident level. The percentage of students 
within each writing achievement level 
range for participating jurisdictions in 2002 
is presented in figure 2.8 for grade 4 and in 
figure 2.9 for grade 8. The shaded bars 
represent the proportion of students in 
each of the three achievement levels {BasiCj 
Prvfident, and Advanced)^ as well as the 
proportion of students who performed 
below die Basic level. Each population of 
students is aligned at the point where the 
Prvfident level begins; scanning down the 
horizontal bars to examine the data on the 
right allows comparison with the percent- 
age of students who were at or above 
Proficient, jurisdictions are listed in the 
figures in three clusters based on a statisti- 
cal comparison of the percentage of 
students at or above Prvfiident in each 
jurisdiction with the national percentage 
of public-school students at or above 



Proftdent. The jurisdictions in the top 
cluster of each figure had a higher per- 
centage of students who were at or above 
die Pwfident level compared to the nation. 
The percentages of students in jurisdic- 
tions clustered in the middle were not 
found to differ significantly from the 
national percentage. Jurisdictions in the 
bottom cluster had percentages lower 
than the national percentage. Within each 
cluster, jurisdictions are listed alphabeti- 
cally. 

Figure 2.8 shows that, at grade 4, 10 
jurisdictions had higher percentages of 
students performing at or above the 
Profident level than the nation, 12 had 
percentages that were not found to differ 
significantly from the nation, and 26 had 
percentages that were lower than the 
nation. 

In figure 2.9, the results for grade 8 
show 10 jurisdictions with higher percent- 
ages of students performing at or above 
die Prvfident level than the nation, 15 with 
percentages that were not found to differ 
significantly from the nation, and 22 with 
percentages that were lower than tiie 
nation. 



ERIC 



30 



CHAPTER 2 • 



NAEP 2002 WRITING REPORT CARD 



48 



Figure 2.8 Percentage of students within each writing achievement level range, grade 4 public schools: By state, 2002 
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The bars below contoin percentages of students in each NAEP writing achievement level range. Each papulation of 
students is aligned at the point where the Prafi'oen/categary begins, so that they may be campored at Pnof/'c/en/and above. 
States are listed alphabetically within three groups: the percentage at or above Prafi'c/en/ was higher than, not found to be 
significantly different from, or lower than the notion. 
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f Indxotes the) the jurisdidbn did not meet one or more of tfie guidebnes for school portidpotum in 2002. 



^ Deportmen! of Defense Domestic D^iendent Qementary aid Secondary Schools. 

NOTE Percentoges moy not odd to 1 00 due to rounding. 

SOURCE Ui. Depojlment of Education, Institute of Education Sciences, National Center for Education Stotisncs, National Assessment of Educational Progress (NAEP), 2002 Writing Assessment. 
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Figure 2.9 Percentage of students within each writing achievement level range, grade 8 public schools: By state, 2002 



The bars below contain percentages of students in each NAEP writing achievement level range. Each population of 
students is aligned at the point where the Prof/c/en/ category begins, so that they may be compared at ProPc/enf and above. 
States are listed alphabetically within three groups: the percentage at or above Prof/c/en/was higher than, not found to be 
significantly different from, or lower than the nation. 
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The percentage of fourth-graders per- 
forming at or above the Proficient level for 
each jurisdiction that participated in the 
2002 assessment is presented in table 2.4. 
The percentage of fourth-graders perform- 
ing at or above tlie Pyvfitciefit level ranged 
from 4 to 49 percent. 



The percentages of eighth-graders at or 
above Proficient for jurisdictions that partici- 
pated in 1998 and 2002 are presented in 
table 2.5. The percentage of eighth-graders 
performing at or above Proficient increased 
since 1998 in 17 jurisdictions, and de- 
creased in 1 jurisdiction. 
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Toble 2.4 Percentoge of students ot or obove Proficient \t\ writing, grode 4 public schools: By stote, 2002 



Grade 4 



Notion (Public) 27 

Alabama 15 

Arizona 1 5 

Arkansas 1 9 

California ♦ 23 

Connecticut 49 

Delaware 35 

Florida 33 

Georgia 23 

Hawaii 22 

Idaho 22 

Indiana 26 

Iowa ♦ 27 

Kansas ^ 21 

Kentucky 27 

Louisiana 14 

Maine 32 

Maryland 30 

Massachusetts 44 

Michigan 19 

Minnesota ^ 29 

Mississippi 1 3 

Missouri 22 

Montana ^ 22 

Nebraska 27 

Nevada 18 

New Mexico 18 

New York ♦ 37 

North Carolina 32 

North Dakota ^ 20 

Ohio 28 

Oklahoma 16 

Oregon 22 

Pennsylvania 29 

Rhode Island 30 

South Carolina 17 

Tennessee * 23 

Texas 29 

Utah 20 

Vermont 32 

Virginia 29 

Washington ^ 30 

West Virginia 1 9 

Wyoming 23 

Other Jurisdictions 

District of Columbia 1 1 

DDESS ’ 25 

DoDDS’ 30 

Guam 9 

Virgin Islands 4 



^ Indicates that the juriscy on did not meet one or more of the guidehnes for school portk^ion in 2002. 

^ Dqxitmefd of Defense Domesik Dependent Dementoy mf Secondary Schook. 

^ Deporlnol of Defense D^ienderls Sduols (Overseers). 

SOliXCE: U.S. Deportment of Educa0or\ Inditute of Educo^ Sciences, Noliomd Center for EikcatkHi Statistics, NaOonol Assessment of Educational iVo^ess (NAEP), 2002 Writing Assessment. 
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Table 2*5 Percentage of students at or above Profidentin writing, grade 8 public schools: By state, 1998 and 2002 





1998 


2002 


Nation (Public) ’ 


24* 


30 


Aioboma 


17 


20 


Arizona 


21 


20 


Arkansas 


13*'** 


19 


California ^ 


20 


23 


Coiarado 


27 


— 


Connecticut 


44 


45 


Delaware 


22 *•** 


35 


Florida 


19*'** 


32 


Georgia 


23 


25 


Hawaii 


15* 


18 


Idaho 


— 


29 


Indiana 


— 


26 


Kansas ^ 




32 


Kentucky 


21 


25 


Louisiana 


12*'** 


18 


Maine 


32 


36 


Maryland 


23 *'** 


35 


Massachusetts 


31 *'** 


42 


Michigan 


— 


24 


Minnesota ^ 


25 


— 


Mississippi 


M 


13 


Missouri 


17 *.** 


27 


Montana ^ 


25 


29 


Nebraska 


— 


32 


Nevada 


17 


16 


New Mexico 


18 


18 


New York ^ 


21 *'** 


30 


North Carolina 


27 *'** 


34 


North Dakota ^ 




24 


Ohio 


— 


38 


Oklahoma 


25 


27 


Oregon ^ 


27*'** 


33 


Pennsylvania 


— 


32 


Rhode Island 


25*.** 


29 


South Carolina 


15*'** 


20 


Tennessee * 


24 


24 


Texas 


31 


31 


Utah 


21 


23 


Vermont 


— 


41 


Virginia 


27* 


32 


Washington ^ 


25*'** 


34 


West Virginia 


18 


21 


Wisconsin ^ 


28 


— 


Wyoming 


23* 


28 


Other Jurisdictions 






American Samoa 




3 


District of Columbia 


11 


10 


DDESS2 


38 


42 


DoDDS^ 


31 *'** 


37 


Guam 




13 


Virgin Islands 


9* 


3 



In(&at 0 s that the jurisdiction did not partk^e or (£d not meet minimum pnficipation guidehnes for reportit^. 
^ Imhcotes that the juriscktion (fid not meet one or more of the guidelines for sdx)o) partlc4ialion in 2002. 



Significantly different from 2002 when only one jurisdictbn or the notion is being exominei 

** SigniOcontly differeiti from 2002 when using a mutt vle-comporison procethire Ixsed on ol jutisi&tions that pcvticquled both years. 

^ Nationd results for the 1 998 assessment ore bcned on the notioid sonple, no! on aggregatd slitie assessment samples. 

^ Deportmoti of Defense Domestic Dependent Oementoiy and Seamdory Schools. 

^ Department of Defense Dependents Schooh (Overseas). 

NOIL Con^orotive performance results may be affected by dun^ in exclusion rotes for students witii dsafailities and bnuled Bigfish proficient dud»tis in the NAB’ sam(des. 

SOURCE: Ui. Department of Education, Institute of Education Sciences, Nafurud Center fu Education Statistics, National Assessment of Educationa! lYogress (NMP), 1 998 and 2002 Writing Assessments. 
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Cross-State/Jurisdiction 
Writing Achievement Level 
Comparisons 

Figures 2.10 and 2.11 display die same 
type of cross-state/jurisdiction compari- 
sons that were presented earlier for scale 
score results, but the performance measure 
being compared in these figures is the . 
percentage of students performing at or 
above the Vwficient level in 2002 for grades 
4 and 8 respectively. 



At grade 4, Connecticut had a higher 
percentage of students performing at or 
above Vroficient than Massachusetts, and 
both had higher percentages tiian the other 
participating jurisdictions. At grade 8, 
Connecticut, Massachusetts, Department 
of Defense domestic schools, and Vermont 
were among the states with the highest 
percentages of students performing at or 
Pwfictent^ but were not found to 
differ significantly from each odier. 
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Figure 2.10 Cross-state comparison of percentage of students at or above ProF/c/enf in writing, grade 4 public schools: 2002 






instructions: Reod down ihe column directly under o jurisdiction nome listed in the heoding ot the top of the chort. Motch the 
shoding intensity surrounding o jurisdiction's obbreviotion with the key below to determine whether the percentage of students ot 
or obove Profic/entin this jurisdiction wos found to be higher thon, not significontly different from, or lower thon the jurisdiction in 
the column heoding. Forexomple, In the column under Vermont, the percentage of students ot or obove Prof/cienMn Vermont wos 
lower thon Connecticut ond Mossochusefts, obout the some os oil the states from New York through lowo, ond higher thon the 
remoining states down the column. ~ 
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1 Jurisdiction had higher percentage than the jurisdiction listed 
at the top of the figure. 



at the top of the figure. 



1 Jurisdiction had lower percentage than the jurisdiction listed 
at the top of the figure. 



f Indicates that the jurisdiction did not meet one or more of the guidelines for school participation in 2002. 
^ Department of Defense Dependents Schools (Overseos). 

^ Department of Defense Domestic Dependent Elementary andSecondary Schools. 



NOIL The between-jurisdlction comparisons take into account sampling and measurement error and that 
each jurisdiction is being compared with every other jurisdiction. Significance is determined by an 
application of a multipleKomparison procedure (see appendix A). 

SOURCE U.S. Department of Education, Institute of Education Sciences, National Center for Education 
Statistics, National Assessment of Educational Progress (NAEP), 2002 Writing Assessment. 
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Figure 2A I Cross*stote comporison of percentage of students ot or obove Proficient \n writing, grode 8 public schools: 2002 



Instructions: Read down the column directly under a jurisdiction name listed in the heading at the top of the chart. Match the 
shading intensity surrounding a jurisdiction's abbreviation with the key below to determine whether the percentage of students at 
or above Profic/entin this jurisdiction was found to be higher than, not significantly different from, or lower than the jurisdiction in 
the column heading. For example, in the column under Delaware, the percentage of students at or above Proficient in Delaware 
was lower than Connecticut, Department of Defense Domestic Schools, Massachusetts, and Vermont, about the some os oil the 
states from Ohio through Texas, and higher than the remaining states down the column. ^ 
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at the top of the figure. 

I I No significant difference detected from the jurisdiction listed 
atthetopofthe figure. 

I I Jurisdiction had lower percentage thon the jurisdiction listed 
at the top of the figure. 



I Indicates that the jurisdiction did not meet one or more of the guidelines for school porticipotion in 2002. 
^ Department of Defense Domestic Dependent Elementary and Secondary Schools. 

^ Department of Defense Dependents Schools (Overseas). 

NOTE: The between-jurisdiction comparisons take into occount sampling and measurement error and that 
each jurisdiction is being compared with every other jurisdiction. Significance is determined by on 
application of a mulliple-comparison procedure (see appendix A). 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education 
Statistics, Notional Assessment of Educational Progress (NAEP), 2002 Writing Assessment. 
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Subgroup Results 

for the Nation and States 

In addition to reporting on the performance of all students, 



NAEP also provides results for a variety of subgroups of 
students for each grade level assessed. The subgroup results 
show not only how these groups of students performed in 
comparison with one another, but also the progress each 
group has made over time. The information presented in 
this chapter is a valuable indicator of how well the nation is 
progressing toward the goal of improving the achievement 
of all students. 

This chapter includes average writing scale scores and 
achievement level results for subgroups of students in the 
nation at grades 4, 8, and 12, and in participating 
jurisdictions at grades 4 and 8. National results are reported 
by gender, race/ethnicity, students’ eligibility for free/ 
reduced-price school lunch, school’s participation in Tide I, 
parents’ highest level of education, type of school, and type 
of school location. Results for participating jurisdictions are 
presented by gender, race/ethnicity, and students’ eligibility 
for free/reduced-price school lunch. Descriptions of these 
subgroups are included in appendix A under “NAEP 
Reporting Groups.” The weighted percentage of students 
corresponding with each subgroup reported in this chapter 
can be found in appendix B. Additional subgroup results 
for each jurisdiction that participated in the NAEP writing 
assessment are available on the NAEP web site (http:// 
nces.ed.gov/ nationsreportcard/ naepdata). 
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Differences in students’ performance on 
die 2002 writing assessment between 
demographic subgroups and across years 
for a particular subgroup are discussed only 
if they have been determined to be statisti- 
cally significant. The reader should bear in 
mind that the estimated scale score for a 
subgroup of students does not reflect the 
entire range of performance within that 
group. Differences in subgroup perfor- 
mance cannot be ascribed solely to stu- 
dents’ subgroup identification. Average 
student performance is affected by the 
interaction of a complex set of educa- 
tional, cultural, and social factors not 
discussed in this report or addressed by 
NAEP assessments. 



Performance of Selected 
Subgroups for the Nation 

Gender 

In recent years, educators and researchers 
have produced a number of studies docu- 
menting gender differences in writing 
performance. Some of these studies focus 
on qualitative differences between the 
writing produced by boys and tliat pro- 
duced by girls.^ Other studies examine 
quantitative differences in language use and 
writing performance by gender.“ 

Results from the NAEP writing assess- 
ment reflect similar patterns in perfor- 
mance between male and female students. 

Figure 3.1 presents national average 
writing scale scores for male and female 
students in grades 4, 8, and 12, across 
assessment years. In 2002, female students 
outperformed their male peers at all three 
grades. The average scores of male and 
female fourth-graders and eighth -graders 
were higher in 2002 than in 1998; however, 
at grade 12 the average scores for male 
students declined, while no change in the 
average scores for female students was 
detected during the same interval. 



^ Levine, T, and Geldnian-Caspar, Z. (1996). Informal Science Writing Produced by Boys and Girls: Writing 
Preference and Quality British YLducaiional Research Journal 22(4)^ 421-439. 

Peterson, S, (2001), Gender Identities and Self-Expression in Classroom Narrath^e Writing. Language Arts 75(5), 
451-457. 

Tliomas, P. (1994). Writing, Reading, and Gender. Gifted Hducatio/i International, P(3), 154—158. 

^ Ashmore, R., and Sliields, C. (2002). Tlie Acliievement Gap. A Comparison of Anglo and Navajo Student Writing 
Samples. Planning and Changing, d3(V), 91—105. 

Berninger, V. W, and Fuller, F. (1992). Gender Differences in Orthographic, Verbal and Compositional 
Fluency: Imphcations for Assessing Writing DisabiUties in Primary Grade Cliildren. Journal of School Psychology, 
30(4), 363-382. 

Hoff Sommers, C. (2000). The If^ar Against B^s. New York: Simon and Schuster 
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Figure 3.1 Average writing scale scores, by gender, grades 4, 8, and 1 2: 1 998 and 2002 



Grades 4, 8, and 1 2 



Grade 4 Grade 8 Grade 1 2 






• Female 
■ Male 



Significantly different from 2002. 

SOURCE: U5. Department of Education, Institute of Education Sciences, Naliond Center for Education StotistiQ, Nafionol Assesanent of Educational Progress (NAEP), 1 998 and 2002 Writing Assessments. 



Another way to compare student perfor- 
mance is to determine whether there is a 
difference or ^‘gap” between the subgroup 
average scores and whether that gap 
increases or decreases between assessment 
years. The scale score gaps between male 
and female students are presented in figure 
3.2. In 2002, the difference in average 
writing scale scores favoring females was 



17 points at grade 4, 21 points at grade 8, 
and 25 points at grade 12. Between 1998 
and 2002, no significant change was 
detected in the scale score gap between 
male and female students at grades 4 and 8; 
however, a significant increase in the gap 
between males and females was noted at 
grade 12. 
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Figure 3.2 Gaps in average writing scale scores, by gender, grades 4, 8, and 1 2: 1 998 and 2002 



Grades 4, 8, and 1 2 



Grade 4 



Female average score minus male average score 

Grades Grade 12 
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2002 
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* Significant!/ (Afferent from 2002. 

NOTt Sure gaps ore coyoled bosed on differences between unrounded overage sode scores. 

SOURCE: U.S. Depotlment of Educotion, Institute of Education Sciences, Nalioncd Center for Education Statistics. Notional Assessment of Education^ Progress (NAEP), 1 998 and 2002 Writing Assessments. 



Table 3.1 displays achievement level 
information for the national sample of 
fourth', eighth', and twelfth-graders both 
as the percentages of male and female 
students within each achievement level 
range and as the percentages of male and 
female students at or above the Basic and 
Vroftcient levels. At grade 4, tlie percentages 
of male and female students performing at 
or above Basic and at or above Prvftdent 
were higher in 2002 than in 1998. At grade 
8, the percentages of both males and 
females performing at or above Proficient 
increased since 1998. At grade 12, the 



percentage of male students performing at 
or above Basic lower in 2002 than in 
1998. While the percentage of female 
twelfth-graders performing at or above 
Pioficient increased since 1998, no change in 
the percentage of male students performing 
at or above Proficient was observed for the 
same time period. 

Higher percentages of female students 
performed at or above the Basic and Profit^ 
dent levels, and at Advanced^ than their male 
peers in all three grades. 
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